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7—i Abstract. In nonextensive statistical mechanics, two kinds of definitions have been 

^ considered for expectation value of a physical quantity: one is the ordinary definition 

in 

(N 

^J- and the other is the normalized ^-expectation value employing the escort distribution. 

O 

Since both of them lead to the maximum-Tsallis-entropy distributions of a similar type, 

-i— » 

S it is of crucial importance to determine which the correct physical one is. A point is that 

q the definition of expectation value is indivisibly connected to the form of generalized 

O 

• • 

relative entropy. Studying the properties of the relative entropies associated with these 

X 

c3 two definitions, it is shown how the use of the escort distribution is essential. In 

particular, the axiomatic framework proposed by Shore and Johnson is found to support 
the formalism with the normalized ^-expectation value and to exclude the ordinary 
expectation value in nonextensive statistical mechanics. 
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I. INTRODUCTION 

Nonextensive statistical mechanics [1-4] pioneered by Tsallis [5] offers a consistent 
theoretical framework for the studies of complex systems in their nonequilibrium 
stationary states, systems with (multi)fractal and self-similar structures, long-range 
interacting systems, anomalous diffusion phenomena, and so on. The worked examples 
are dynamical systems at the edge of chaos [6-9], lattice Boltzmann models [10], 
magnetism of colossal magnetoresistance manganites [11], high-energy processes [12- 
16], cosmic rays [17], cellular aggregates [18], Levy flights [19,20], semiclassical 
dynamic in optical lattices [21], astrophysics and self-gravitating systems [22,23], 
econophysical problems [24,25], kinetics of charged particles [26], Internet traffic [27], 
earthquakes [28,29], and complex networks [30-32] 

In spite of these successes, still there remain some fundamental questions to be 
answered in the theory. One of them, which is of extreme importance, is concerned with 
the definition of expectation value. The most widely employed definition in 
nonextensive statistical mechanics is the normalized ^-expectation value [33], which is 
given by 

u ^ = <H>q = ^P i e i , (1) 

P= <"<>' ,2) 

where P j is termed the escort distribution [34] associated with the basic distribution p i 
and H denotes a physical random variable (e.g., the system energy) with its z'th value 



e, . The index q is taken to be positive. However, in the literature, there is an opinion 
that the ordinary expectation value 

<«*> = </*> = £ (3) 

should be used also in nonextensive statistical mechanics. In Ref. [35], it has been 
shown that, for a class of power-law distributions, only the normalized ^-expectation 
value is consistent with the method of steepest descents for (micro)canonical ensembles, 
but the situation remains unclear for the other class of distributions with compact 
supports. 

Here, we address ourselves to the problem of choice of expectation value in 
nonextensive statistical mechanics. Our procedure is to examine the properties of the 
generalized relative entropies associated with the aforementioned two kinds of 
definitions. We shall see how the formalism with the normalized ^-expectation value is 
superior to that with the ordinary expectation value. 

The paper is organized as follows. In Sec. II, the ordinary and normalized q- 
expectation values are reexamined. A novel geometric aspect of the maximum entropy 
principle is also pointed out, there. In Sec. Ill, two different kinds of the generalized 
relative entropies associated with these definitions of expectation value are considered 
and their properties are studied. In Sec. IV, an axiomatic approach to the issue is 
developed, and is shown to support the normalized ^-expectation value, excluding the 
possibility of using the ordinary expectation value in nonextensive statistical mechanics. 
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Sec. V is devoted to concluding remarks. 



II. ORDINARY AND NORMALIZED # -EXPECTATION VALUES 

Before discussing the problem of expectation value, first we wish to point out a 
geometric aspect of the maximum entropy principle, which will be used later. The idea 
was inspired by the work of Bashkirov [36]. Consider a functional defined in the 
space E of probability distributions. Two operations on E are of interest: one is 
translation and the other is dilatation. The corresponding generators are given by 

T,= * (4) 
°Pi 

i 8 Pi 

respectively, where p,-eE. Clearly, they satisfy the following closed algebra: 
[T i9 Tj] = 0, [T i ,D\ = T i , [D,D]-Q. Invariance of the functional <I> under these 
operations implies 

T^ = 0, (6) 
D4> = 0. (7) 
If $ is an entropic functional, the solution to these equations yields the maximum 
entropy distribution. A point to be noticed is that the dilatation operation is constrained 
by the normalization condition 

y>,-i=o. (8) 



This, in turn, determines the value of the associated Lagrange multiplier. 



Now, let us apply this method to the Tsallis entropy indexed by q [5] 



SAP] = 



l-q 



(9) 



Here and hereafter, the Boltzmann constant is set equal to unity for the sake of 



simplicity. 



If the constraint is imposed on the ordinary expectation value, then the functional to 



be optimized is 



( \ ( \ 

(ord) 



&° rd) [p;a,l3] = S q [p]-(X J>,-1 - fi J>,£,-t/ 

V i ) V i 



where a and f5 are the Lagrange multipliers. Eqs. (6)-(8) are found to give 



(10) 



l-q v ' 



a = 



l-q 



l + (l-q) Sf d) ]-/?£/ (ord) , 



(ID 



(12) 



where 5* ord) and f/ (ord) are the values of S„ and [/ <ord) calculated in terms of the 



maximum entropy distribution p ; (ord) , respectively. From these equations, it follows that 



+ (l-q)~S^] 



i/(?-D 



P' (£, -f/ (ord) ) 



ll(q-l) 



(13) 



where 



(14) 



I, (pry 

On the other hand, if the normalized ^-expectation value is employed, the corresponding 



functional reads 



5 



V i ) 



-u 



(nor) 



(15) 



Here, we are using the same notation for the Lagrange multipliers as in Eq. (10), but it 
will not cause any confusion. The operator D acting on the third term on the right-hand 
side trivially vanishes since this term is manifestly invariant under the dilatation. Eqs. 



(6)-(8) give rise to 



(16) 



a = 



l-q 



1 + 



(17) 



where 



P 



(18) 



L (pry 

5* nor) and f/ (nor) are the values of S„ and U (noi) calculated in terms of the maximum 



entropy distribution p ; (nor) . Eqs. (16) and (17) lead to 



~ (nor) _ 1 
*i ^(nor) 



(l-q)P*(s i -U^)] 



(19) 



7 (nor) _ 



1 + 



1/(1-9) 



= E[l-(l-«)i8-(c < -^->)] 



1/(1-9) 



(20) 



Eqs. (13) and (19) are quite similar to each other, but the signs of the exponents are 



opposite. 



An important point is that with both of the definitions of expectation value the 



following thermodynamic relations hold: 




p, 



(21) 



du (0Id) 




P, 



(22) 



du (nm) 



which may indicate that the thermodynamic Legendre-transform structure is established 
in both cases. However, it is still an open problem in nonextensive statistical mechanics 
if [3 is the physical inverse temperature [37]. 

It is clear that, in the limit q — > 1, the Tsallis entropy in Eq. (9) tends to the 
Boltzmann-Gibbs-Shannon entropy S[p] = —^. p, lnp ( ., and accordingly both of the 
distributions in Eqs. (13) and (19) converge to the familiar Boltzmann-Gibbs 
distribution p i ~ exp (-/J £, ) . 

III. GENERALIZED RELATIVE ENTROPIES 

Relative entropy plays a fundamental role for comparing two distributions. There 
exist two different kinds of the generalized relative entropies in nonextensive statistical 
mechanics. One is given by [38] 



where r i is a reference distribution. In what follows, we shall look at the properties of 




(23) 



and the other is [39-41] 




(24) 
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these two quantities in detail. In particular, in subsection b, we shall see that I q [p\\r] 
and K q [p\\r] are the relative entropies associated with the ordinary expectation value 
and the normalized g-expectation value, respectively. 



a. Correspondence relation and nonnegativity 



First of all, we notice that, in the limit g— >1, both / [pllr] and K [p\\r] tend to 



the Kullback-Leibler relative entropy 



H[p\\r] = Y, P> — 



(25) 



The following expression is often used for this quantity: 



H[p\\r] = ^-Y.^y^ 

(IX ; 



l-x 



(26) 



K [p\\r] in Eq. (24) is obtained by replacing the differential operator by the Jackson q- 



differential operator [42] 



(27) 



X->1 



where D q f(x) = [f(qx) - f(x)] I [x(q - 1)], which satisfies the ^-deformed Leibniz 
rule D q [f(x)g(x)] = [D q f(x)]g(x) + f(x)D q [g(x)] + x(q- l)D q [f(x)]D q [g(x)] and 



converges to the ordinary differential in the limit q—>l. 



No such simple correspondence is known to exist between H[p\\r] and I [p\\r]. 



One could, however, still dare to write 



H[p\\r] = 



dG(x) 
dx 



(28) 



g(x)=-X [w -(*■,)*]-£ (Pi-^wy- 1 , (29) 

and accordingly 

/J/7llr] = ? D 9 G(x)| (30) 

Like H[p\\r], I g [p\\r] and ^[pllr] are nonnegative and vanish if and only if 
p i = r i (Vi). This can be seen as follows. Regarding I g [p\\r], it is convenient to 

employ the integral representation [38] 

Pi 

= Jd*^- 1 -^)'- 1 ], (31) 

from which nonnegativity follows immediately. On the other hand, K q [p\\r] can be 
rewritten as 

K q [p\\r] = Pl [l - (r, / p,.) 1 "*]. (32) 

Noticing (1 - x'~ ? ) / (1 - q) > 1 - x for jc> and q>0 with the equality for x = l, 
K [p\\r] is also seen to be nonnegative. 



b. Free energy difference 

Let us discuss the physical meanings of I g [p\\r] and K q [p\\r], in particular, their 
relevance to the definitions of expectation value. For this purpose, we take the 
maximum entropy distributions as the reference distributions. The exponents of the 
maximum entropy distributions in Eqs. (13) and (19) together with the dependencies of 
I q [p\\r] and K q [p\\r] on r, should be noticed. 
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Putting r i = p ( (ord) in Eq. (23), we have 

I q [p\\p (ord) ] = p(F^-F^ d) ), (33) 

where 

F(ord) _ jy(ord) J_ ri p(ord) _ jy(ord) J_ n (ord) f^A") 

? 9 ' 9 (3 q 

On the other hand, substituting r, = p ( (nor) into Eq. (24), we obtain 

K q [pW~p^] = f (f,<~> ~ F^), 05) 

LAP' ) 



where 



P=PZW. (36) 



^ (nor) _ jj(nor) J_ g p(nor) _ jj(nor) J_g(nor) (37) 



Eqs. (33) and (35) show that / ? [/?llr] and K q [p\\r] are essentially the free energy 
differences and, therefore, are recognized as the generalized relative entropies 
associated with the ordinary expectation value and the normalized ^-expectation value, 
respectively. We also mention that the quantum mechanical counterpart of K q [p\\r] has 
recently been employed to prove the second law of thermodynamics [43]. 



c. Convexity 

Convexity is one of the most important properties to be fulfilled by relative entropy. 
Taking the second-order derivatives of I g [p\\r] with respect to the arguments, one 
finds that it is convex in p t , but not in r i . 
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On the other hand, like the Kullback-Leibler relative entropy, K [p\\r] is seen to be 



jointly convex (see Ref. [44] in the quantum mechanical case): 



K 



Ha^aPia) La^M ^ L a K K q iP (a) 11 ^ . 



(38) 



where A a > and ^ a = 1- This property is stronger than individual convexity in 



Pi and r r 



d. Composability 

Finally, we notice that, like the Kullback-Leibler relative entropy, K q [p\\r] is 
"composable" [45], but I g [p\\r] is not. In fact, for factorized joint distributions of a 
composite system (A, B), p u (A, B) = p a)i (A) p (2)j (B) and r tj (A, B) = r (l)i (A) r (2)j (B), 

K q \-Pm P(2) llr d) r (2>] y ields 

r (l) r (2)J 

= K q [p m \\r m ] + K g [p (2) \\ 

r (2)i 

+(q-l)K q [p m \\r m ]K q [p m \\r (2) l (39) 
whereas no such closed relation exists for I g [p m p (2) Wr (l) r (2) ]. Eq. (39) has its origin in 
the ^-deformed Leibniz rule satisfied by the Jackson ^-differential operator. 



IV. SHORE- JOHNSON AXIOMS 

In the preceding section, we have seen how K q [p\\r] associated with the normalized 
^-expectation value has the properties, which are more favorable than those of I q [p\\r] 
corresponding to the ordinary expectation value. In this section, we discuss that the 
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choice of K q [p\\r] is in fact supported by a set of axioms. 

About a quarter a century ago, Shore and Johnson [46] have proposed the axioms for 
minimum cross-entropy (i.e., relative entropy) principle. They are composed of the 
following five statements (presented in a nonabstract manner): 

• Axiom I (Uniqueness): If the same problem is solved twice, then the same answer is 
expected to result both times. 

• Axiom II (Invariance): The same answer is expected when the same problem is solved 
in two different coordinate systems, in which the posteriors in the two systems should 
be related by the coordinate transformation. 

• Axiom III (System independence): It should not matter whether one accounts for 
independent information about independent systems separately in terms of their 
marginal distributions or in terms of the joint distribution. 

• Axiom IV (Subset independence): It should not matter whether one treats independent 
subsets of the states of the systems in terms of their separate conditional distributions or 
in terms of the joint distribution. 

• Axiom V (Expansibility): In the absence of new information, the prior (i.e., the 
reference distribution) should not be changed. 

These axioms are extremely natural, and should be fulfilled in any physical 
situations. 

For the Tsallis entropy in Eq. (9), the axioms and uniqueness theorem are known in 
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the literature [47]. In contrast to this fact, the above set of axioms is quite general and 
not very restrictive, and therefore does not uniquely determine the definition of the 
relative entropy. However, still it can be shown [46] that the relative entropy 7[/>llr] 
with the prior r i and the posterior p i satisfying the axioms I-V has the following form: 

J[pWr] = Y, PiHPi/r,), (40) 

i 

where h(x) is some function. 

At this juncture, it is crucial to recognize that the function h(x) surely exists for 
K q [p\\r\. 

h(x) = -^-(l-x"- 1 ), (41) 
1-q 

whereas I g [p\\r] cannot be recast to the form in Eq. (40). Therefore, we conclude that 
the Shore-Johnson axioms support the normalized ^-expectation value and exclude the 
possibility of using the ordinary expectation value in nonextensive statistical mechanics. 



V. CONCLUDING REMARKS 

We have discussed two kinds of definitions of expectation value in nonextensive 
statistical mechanics, that is, the ordinary expectation value and the normalized q- 
expectation value. To determine which the correct definition is, we have studied the 
corresponding generalized relative entropies. It was found that the generalized relative 
entropy associated with the normalized ^-expectation value has the properties, which 
are superior to those associated with the ordinary expectation value. More decisively, 
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the Shore-Johnson axioms are shown to select the formalism with the normalized q- 
expectation value and to exclude the possibility of using the ordinary expectation value 
in nonextensive statistical mechanics. 
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